Bayesian Optimal Control of Smoothly Parameterized Systems: The Lazy Posterior Sampling Algorithm

摘要

We study Bayesian optimal control of a general class of smoothlyparameterized Markov decision problems. Since computing the optimal control iscomputationally expensive, we design an algorithm that trades off performancefor computational efficiency. The algorithm is a lazy posterior sampling methodthat maintains a distribution over the unknown parameter. The algorithm changesits policy only when the variance of the distribution is reduced sufficiently.Importantly, we analyze the algorithm and show the precise nature of theperformance vs. computation tradeoff. Finally, we show the effectiveness of themethod on a web server control application.